The present invention relates to an image processing method and apparatus for generating an image from the viewpoint of an observer on the basis of images obtained from a plurality of viewpoints.
Conventionally, as an apparatus for stereoscopically displaying images viewed from a plurality of viewpoints, a stereoscopic display, a lenticular display, and the like are known. The stereoscopic display displays images which are obtained from two cameras, and which are alternately switched at a high speed. An observer can stereoscopically observe the displayed image using shutter glasses which is synchronized with the switching operation of the images or polarization glasses. In the lenticular display, images from, e.g., four cameras are re-arranged in units of pixels, and by adhering a lenticular sheet to the front surface, an image from four viewpoints can be stereoscopically expressed.
However, with the above-mentioned conventional stereoscopic display, only a stereoscopic image in the photographing directions of the cameras upon photographing of an image can be observed. More specifically, since the two cameras are fixed in position to photograph an object, an observer observes only an identical image even when he or she moves his or her viewpoint (the positions of the eyes), and viewpoint movement on the observer side is not reflected in a displayed image, resulting in an unnatural image. On the other hand, the lenticular display can cope with movement of the viewpoint of the observer in the right-and-left direction, but cannot cope with continuous viewpoint movement since it allows only sequential observation of discrete images from the cameras. In addition, in the lenticular display, the viewpoint cannot be moved in the back-and-forth direction, i.e., a direction along the line of sight. A viewpoint movement in the back-and-forth direction is realized only when a stereoscopic view is achieved based on images generated by computer graphics. However, such images generated by computer graphics are special ones since they are simple and the coordinate values, in a corresponding three-dimensional space, of all the points in the images are known. For this reason, the possibility of a viewpoint movement in the back-and-forth direction has not been examined so far when images photographed by the cameras are to be stereoscopically observed.